CLAIMS 
We claim: 

1. A business method comprising the steps of: 

monitoring one or more multimedia items accessed by a user, each multimedia item having two 
5 or more disparate modalities, the disparate modalities being at least one or more visual 
modalities and one or more textual modalities; 

creating a visual feature vector for each of the visual modalities and a textual feature vector for 
each of the textual modalities; 
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concatenating the visual feature vectors and the textual feature vectors into a unified feature 



m 

ylO vector; 



^ categorizing each of the multimedia items by categorizing the respective unified feature vector; 
and 



assembling a user profile based on the categorization. 

2. A business method, as in claim 1, further comprising the step of: 

1 5 using the user profile to match one or more multimedia items stored in one or more databases. 

3. A business method, as in claim 2, where one or more of the databases are part of a computer 
that is connected to one or more networks. 

4. A business method, as in claim 3, where the networks include any one or more of the 
following: an Internet, an intranet, an extranet, a corporate network, a government network, a 

20 infrared network, and a radio frequency network. 
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5. A business method, as in claim 1, where the categories include any one or more of the 
following: a product, a service, an interest, a retail item, a hobby, a food item, an item of 
clothing, a travel package, a vacation destination, a financial product, a business partner, a 
business interest, a medical product, a commercial, and a social interest. 

5 6. A business method, as in claim 1, where the category includes any one or more of the 
following services: consulting, legal, real estate, medical, technical, physical training, diet, 
cosmetic, fashion, governmental, automotive, design, architecture, personal assistants, games, 
on-line games of chance, dating services, and landscaping. 

3 .... 
gj 7. A business method comprising the steps of: 

UJ 10 scanning one or more multimedia items in a database, each multimedia item having two or more 

S5S 

^ disparate modalities, the disparate modalities being at least one or more visual modalities and 



m 



one or more textual modalities; 



m creating a visual feature vector for each of the visual modalities and a textual feature vector for 



O 



each of the textual modalities; 

15 concatenating the visual feature vectors and the textual feature vectors into a unified feature 
vector; 

categorizing each of the multimedia items by categorizing the respective unified feature vector; 
and 



creating one or more indices of the database based on the categorization. 
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8. A business method, as in claim 7, where the database resides on any one or more of the 
following: a network server, a web site, a personal computer, a server farm, and a network disk 
array. 

9. A business method, as in claim 7, further comprising the step of: 
5 making a business decision based on the classifications. 

10. A business method, as in claim 9, where the categorization is used to organize a collection of 
the multimedia items into a database where the multimedia items are retrievable based on an 
annotation of the classification. 

5 1 1 . A business method, as in claim 10, where the multimedia items that are retrieved are used as 

fi 

gj 10 a response to a query of a search engine. 



M= 12. A business method, as in claim 9, where the multimedia item is a multimedia e-mail and the 

m 

l_ multimedia e-mail is routed based on one or more of the categories assigned to the multimedia 

□ 



13. A business method, as in claim 12, where the multimedia e-mail is routed to any one or more 
15 of the following: a sender, a folder, a person, a personal folder, a corporate folder, and a 

corporate department. 

14. A business method, as in claim 13, where the multimedia e-mail is multiplied before being 
routed. 

15. A business method comprising the steps of: 



m 



e-mail. 
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scanning one or more multimedia items in a database, each multimedia item having two or more 
disparate modalities, the disparate modalities being at least one or more visual modalities and 
one or more textual modalities; 

creating a visual feature vector for each of the visual modalities and a textual feature vector for 
5 each of the textual modalities; 

concatenating the visual feature vectors and the textual feature vectors into a unified feature 
vector; 

categorizing each of the multimedia items by categorizing the respective unified feature vector; 

□ 

© comparing one or more of the unified feature vectors to one or more other feature vectors; 

m 

& 10 and 

jjt making a decision based on the comparison. 

p 1 6. A business method, as in claim 1 5, where the multimedia items are digital multimedia copies 

5 

H and the feature vectors represent restricted multimedia items and the comparison determines if 

fn 

D the digital multimedia copies are similar to the restricted multimedia items and the decision is 
15 that the digital multimedia copies are subject to a second restriction. 

17. A business method, as in claim 16, where the second restriction includes any one or more of 
the following: a copyright restriction, a trademark restriction, intellectual property restriction, 
parental guidance restriction, common decency restrictions, user-defined restrictions. 
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18. A business method, as in claim 16, where the digital multimedia copies are accessible over 
the Internet. 

19. A business method, as in claim 15, where the comparison indicates a degree of similarity and 
the decision is that the multimedia item contains a known content. 
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